Message-based communication streams are generally used in telecommunication networks, big data analytics frameworks, etc., to handle incoming data in real time. Specifically, in conventional systems, real time processing is done on each input message, and the message is dropped thereafter.
In normal operational circumstances a well-dimensioned system can cope with the input message rate. For temporarily buffering messages, a message queue may be provided. However, any unusual conditions such as temporary large delays in processing for a single message (e.g., due to some unexpected background waiting time, like database look-up) or failure/restart of a message processing element can result in input message queuing beyond the dimensioned queue sizes. In this case, the processing element will not receive some of the input messages at all. Such a situation may cause service disruption.
More generally, there are several reasons why a processing element may want to replay input messages that were transmitted on the stream at some past points in time. However, the nature of known message streams is such that they can only be read once; therefore such replaying is usually not possible.